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Instant Node.js Starter 


Welcome to Instant Node.js Starter. This book has been especially created to 
provide you with all the information that you need to get up to speed with 
Node.js, a platform for building fast and scalable network applications. You will 
learn the basics of nodes, get started with building your first Node.js HTTP server, 
and learn about the five main building blocks of Node.js: modules, callback 
functions, the event emitter, streams, and NPM. 


This book contains the following sections: 


So what is Node.js? helps you find out what Node.js actually is, what you can do 
with it, and why it's so great. 


Installation teaches you how to download and install Node.js with the minimum 
fuss and then set it up so that you can use it as soon as possible. 


Quick start will show you how to start building an HTTP server using Node.js. 


Top 5 features you need to know about will teach you how to perform five tasks 
involving the most important features of Node.js. By the end of this section, 
you will understand and be able to create modules, install third-party modules 
via NPM, and use the three main patterns of Node.js: callback functions, event 
emitters, and streams. 


People and places you should get to know provides you with many useful links 
to the project page and forums as well as a number of helpful articles, tutorials, 
blogs, and the Twitter feeds of Node.js super-contributors. This is vital, because 
every open source project is centered around a community. 
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So, what is Node.js? 


Node.js is an open source platform that allows you to build fast and scalable network 
applications using JavaScript. Node.js is built on top of V8, a modern JavaScript virtual 
machine that powers Google's Chrome web browser. 


Node.js uses an event-driven, non-blocking I/O model that makes it lightweight and efficient. 
Node.js can handle multiple concurrent network connections with little overhead, making it 
ideal for data-intensive, real-time applications. 


With Node.js, you can build many kinds of networked applications. For instance, you can use it 
to build a web application service, an HTTP proxy, a DNS server, an SMTP server, an IRC server, 
and basically any kind of process that is network intensive. 


You program Node.js using JavaScript, which is the language that powers the Web. JavaScript is 
a powerful language that, when mastered, makes writing networked, event-driven applications 
fun and easy. 


Node.js recognizes streams that are resistant to precarious network conditions and misbehaving 
clients. For instance, mobile clients are notoriously famous for having large latency network 
connections, which can put a big burden on servers by keeping around lots of connections 

and outstanding requests. By using streaming to handle data, you can use Node.js to 

control incoming and outgoing streams and enable your service to survive. 


Also, Node.js makes it easy for you to use third-party open source modules. By using Node 
Package Manager (NPM), you can easily install, manage, and use any of the several modules 
contained in a big and growing repository. NPM also allows you to manage the modules your 
application depends on in an isolated way, allowing different applications installed in the same 
machine to depend on different versions of the same module without originating a conflict, for 
instance. Given the way it's designed, NPM even allows different versions of the same module 
to coexist in the same application. 
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Installation 


To install Node.js on a Windows or Macintosh machine, go to the http: //Nodejs. org website 
and click on the INSTALL button. 


eo0o B node.js 





Œ |) nodejs.org 


Node.js is a platform built on sily building fast, s ble network 
applications. Node.js uses an event-driven, non-blocking I/O model that mz it lightweight and 


efficient, perfect for data-intensive real-time applications that run across distributed devices. 


Current Version: v0.10.5 





The browser will then ask you if you wish to download a file coming from the Nodejs. org 
website, to which you should agree. 
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Once the file is downloaded, execute it, launching a graphical installer as shown in the 
following screenshot: 





800 @ Install Node a 


Welcome to the Node Installer 





This package will install node and npm into /usr/local/bin 
8 Introduction 


O License 

O Destination Select 
@ Installation Type 
@ Installation 


O Summary 











Go Back | Continue 











When you click on Continue, the installer will prompt you to accept the Software 
License agreement. 


To continue installing the software you must agree to the terms 
of the software license agreement. 


Click Agree to continue or click Disagree to cancel the installation 
and quit the Installer. 


| Read License | [ Disagree | | Agree 
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If you agree with the terms, click on the Agree button (if you run a Mac OS) or check an 
agreement checkbox and click on the Next button (if you run a Windows OS); then click on 
the Install button that follows it. On some systems, the installer will prompt you for a system 
username and password that gives you permission to Install Node globally. 





© Install Node x 


The installation was completed successfully. 


Node was installed at 
8 Introduction 


AN Beer /usr/local/bin/node 


O Destination Select npm was installed at 


O Installation Type Jusr/local/bin/npm 


O Installation 
Make sure that /usr/local/bin is in your $PATH. 
O Summary 


Go Back Close | 











After Node is installed in your system, the installer will show you a success screen. The Mac OS 
installer will state where both the Node and the NPM executables were installed. 


If you don't have a Windows or Macintosh machine, Node also installs on the various Unix and 
Linux OS distributions using the appropriate package manager. The Node.js project maintains 
a wiki page with the installation guide for some distributions on the following page: 


https://github.com joyent/Node/wiki/lInstalling-Node.js-via-package- manager 


Some distributions may not keep the latest stable Node.js version available. If that's the case 
or if you simply don't have another option, you can always download, build, and install Node. 
js from the source code. For that, head out to the download page of the Nodej s. or g website 
(http:// Nodejs.org/downl oad/ ) and click on the Source Code icon. That will start the 
download of the source code. 
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Download the Node.js source code or a pre-built installer for your platform, and start 


developing today. 


Current version: v0.8.21 





node-v0.8.21-x86.msi node-v0.8.21.pkg node-v0.8.21.tar.gz 


Once the source code tarball is downloaded, go to this wiki page: 
https://github.com/joyent/Node/wiki/Installation. 


This is a wiki page maintained by the Node.js project that explains what further steps you 
need to take to compile and install Node from the source code. 


Once you have Node.js installed on your machine, you can certify that you have it available 
by opening a shell prompt and typing Node followed by Enter. That should open the Node. 
js Read-Eval-Print Loop (REPL), which is a Node interactive console to which you can send 
JavaScript commands. 





8090 1. node e 
Last login: Tue Mar 5 14:09:06 on ttys000 
pedroteixeira@Pedr A : 

+ node 


> [] 
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You can then try and write something to the console by typing the following command: 
console. log('Hello World!'); 


Once you hit Enter, you will see the Hello World! string written out to the console. You will 
also see undefined written below that. This is because the REPL will print the value of the 
expression you enter there. In this case, this means thatthe cons ol e. | og function call 
returnsundefined. 





800 1. node 


pedroteixeira@Pedros-Ma 

+ node 

> console. log('Hello World!"); 
Hello World! 

undefined 

>i 
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Quick start 


Now that we have a working Node installation in place, we will see how easy it is to create and 
launch a new server with just a few keystrokes. 


Step 1 — Creating a Hello World HTTP server 


Let's then create our first Hello World server in Node.js. For that, create a file named 
hello world http server.js with the following content: 


var http = require('http'); 

var server = http. createServer(); 

server. on('request', function(reg, res) { 
res. writeHead(200, {'content-type': ‘text/plain'}); 
res. write(" Hello World!'); 
res. end(); 


H; 


var port = 8080; 
server. listen(port); 
server. once('listening', function() { 
console. log('Hello World server listening on port %d', port); 


Downloading the example code 


M You can download the example code files for all Packt books you have 
Q purchased from your accountat http://www. packt pub. com. If you 
purchased this book elsewhere, you can visithtt p: // www. packt pub. 
com support and register to have the files e-mailed directly to you. 


Step 2 - Launching the server 


Once you have this file saved, launch it from the command-line prompt by using the Node 
executable, like this: 


$ Node hello world http server.js 
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Once you press Enter, it should immediately output this: 





80900 1. node é 
pedroteixeira@Pedros- k-Pro: 
(master) 
+ node hello_world_http_server.js 
Hello World server listening on port 8080 


[l 





Now you can connect to it using a web browser, pointing the browser to 
http://localhost: 8080. 


You should then see the Hello World! string rendered in the browser as shown in the following 
screenshot. Congratulations! You have successfully connected to your first Node.js HTTP server! 
Now that you've done that, we can explain to you step by step the code that you just created. 





800 localhost:8080 x e 


= Cc localhost:8080 





Hello World! 











The first line is: 
var http = require('http'); 


Here, you are requiring the Node core ht t p module. The Node core API is decomposed into 
several modules, and here we are looking up the module named ht t p, specifying this name 
inside ther equi re function call. The calltor equi r e returns the HTTP module value, which 
we then assign to a variable conveniently named ht t p . This variable could have been named 
whatever we wanted to name it; it simply serves the purpose of having a reference to the Node 
HTTP module for the code that follows. 


Then we use this module to create an HTTP server in the following line: 


var server = http.createServer(); 


Mm 
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This line creates an HTTP server that we'll later wire up to make it behave as we want. 
Then we have the following block of code: 


server. on('request', function(reg, res) { 
res.writeHead(200, {'content-type': ‘text/plain'}); 
res, write('Hello World!'); 
res.end(); 


RE 


Here we're binding a function to the serverr equest event. This event gets fired once the server 
receives a new HTTP request. Since you have this function bound to that event, it gets called 

for every request that this server answers, having two objects passed in as arguments: an HTTP 
server request and an HTTP server response. You can then use this last object to reply to 

the client. 


The first thing we do when we get a request is write out an HTTP header specifying the response 
content type. Here we're setting the content type tot ext / pl ai n, as an example: 


res. writeHead(200, {'content-type': ‘text/plain'}); 
Next, we're using the HTTP server response object to write out a string to the browser: 
res. write('Hello World!'); 


After that, we end the response, which is required for the HTTP protocol to end and the browser 
to know that the response has ended: 


res.end(); 


After we are bound to ther equest event, we make the HTTP server listen to a specific 
TCP port: 


var port = 8080; 
server. l|isten( port); 


This makes the server available for requests on the TCP port 8080. Once the server is listening 
on that port, the server emits al ist ening event. By listening to that event, we can print out 
a message once the server is available: 


Server.once('listening', function() { 
console. log('Hello World server listening on port %d', port); 


}); 


Here, we're using a variant of thes er ver. on method, namedserver. once, which behaves 
in the same way but only cares about the first time that the event occurs. Since, in our case, 
thel i st ening event will be fired only once, usingser ver. once is more appropriate than 
server.on. 


| 
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Top 5 features you need to know about 


In the last section, we included the ht t p module in the local application by using ther equi re 
function, passing it a name of the module you wish to include. This function can be used for 
including and using any other module that Node. j s comes with. Here is a short list of some 
of the core modules: 

+ net: For creating TCP clients and servers 
ht t p: For creating and consuming HTTP services 
f s : For accessing and manipulating files 
dns: For using the DNS service 
events : For creating event emitters 
stream: For creating streams 


05: For accessing some local operating system statistics 


+++ + 4 +44 


assert:Forassertion testing 


+ Jutil:Formiscellaneous utilities 


You can, for instance, import the f s module into a local variable by running the following 
module in your JavaScript code: 


var fs = require('fs'); 


User modules 


Node.js supports three main types of modules: core modules, user modules, and third-party 
modules. Let's now see how you can use the Node.js module system to create and use modules 
of your own. 


In Node.js, a module is simply a file. To test this, let's create the simplest of modules in a file 
namedsi mpl e_ module.js: 


module. exports = 'ABC'; 
This is a module that simply exports the string ABC. 
Then, in the same directory, create another module namedi mport. js that will use this file: 


var simpleModule = require('./simple module.js'); 
console. log(simpleModule); 


Now, runthei mpor t.j s file using the Node executable by running the following in a 
command-line prompt: 


$ Node import.js 


Mm 
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You should get the following output: 
ABC 
Let's see what we did here. 


Inside thesi mpl e_modul e. js file, you declared that what you wanted to export from that 
module was the ABC string. 


Then, inthe i mport. j s module, you imported that module by requiring it to use a relative 
path(./simple_ module. js). Ifyou give a path instead of a module name, Node.js will look 
for that file and try to load it. If you provide a relative path, Node.js will use that in relation to 
the current file, which in our case resolves to afilenamedsi mpl e_modul e. js inside the same 
directory asi mport.js. 


Now let's create a folder named | i b and create a new module in! i b/ other module. js 
containing the following JavaScript code: 


module.exports = 'DEF'; 

Also, replace the code ini mpor t. j s with the following code: 
var simpleModule = require('./simple_ module.js'); 
console. log(si mpl eModule); 


var otherModule = require('./lib/other_module.js'); 


console. log(otherModule); 
You can see that we're importing the new module using the path relative to the current file. 
You canruntheimport.js file again, like this: 
$ Node import.js 
It should output the following: 
ABCDEF 


You can also use two dots (. . ) to specify parent directories as you would in a normal 
relative path: 


require('../module_a.js'); 
require('../folder/module_b.js'); 
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= You can also use absolute paths, suchas/ path/to/my/module.js, 
Ge but since they're not generally portable between different computers, 
you should avoid them. 


Also, you can omit the. j s extension if you wish to. For instance, thei mport.js file code 
could be transformed to the following without resulting in any difference: 


var simpleModule = require('./simple_module'); 
console. dir(si mpl eModule); 
var otherModule = require('./lib/other_module'); 


console. dir(otherModule); 


Node Package Manager (NPM) 


Node.js has a low-level API that is somewhat a translation and simplification of the Unix 
filesystem and networking API into JavaScript. If you plan to build a complex application, 
doing that solely on top of the core Node.js functionality can be hard and unproductive. 


Fortunately, Node.js has a way of browsing, querying, installing, and publishing third-party 
modules into a central repository, and it's called NPM. NPM stands for Node Package 
Manager, and it consists of two things: 


+ A module repository that is fully browsable, accessible athttps://npmjs.org/ 
+ Acommand-line utility 
The NPM repository contains a vast and growing collection of modules that were built by the 


community. Since that repository is fully browsable and searchable, you can use it to track 
down and inspect modules that may be interesting for building your application with. 


foe Given the sheer number of available modules, a better source of module 
usefulness and quality can be the community forums and IRC channel, or 
á even to inspect the code yourself. 


In NPM, each packaged module has a name, which serves to uniquely identify that module. 
Each module can have one or more versions, and you can specify which one to install. 
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To start watching NPM in action, let's first createa package. j son file. This file serves as a local 
package manifest, contains JSON (an object serialization format that is a subset of JavaScript), 
is placed at the root of your application folder and is, among other things, where you can declare 
what NPM modules your application depends on. Let's create one, then: 


{ 
"name": "my-example-app", 
"version": "0.1.0", 
"dependencies": { 
"request": "*", 
“anos IL, 
"async": "~0.2" 
} 
} 


Once you have saved the contents of package. j son inthe current directory, run the following 
at the command line: 


$ npm install 


Once you have hit the Enter key, NPM will analyze the dependencies of yourpackage. j son file 
and try to download it from the Internet and install it. 


Once NPM finishes, you should have ther equest,nano,andasync modules from NPM 
installed locally. You can then use the np ml s command to list the modules installed in the 
current directory. 





1.0 No README.md file found! 
request 


2.14.0.tgr 
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This last command not only lists the NPM modules installed locally but also the modules that 
those modules depend on, recursively rendering it into this tree. 





80900 1. bash id 


+ npm ls 
npm WARN pac j my-example-app@0.1.0 No README.md file found! 
my-example-appé0.1.0 /Users/pedroteixeira/projects/publications/packt-node/code/04 
H— async@0.2.6 
Fr nano@3.3.7 
H errs@0.2.4 
Hr follow@d.8.@ 
| -— request@2.2.9 
F-r request@2.12.@ 
| Hr form-data@d.0.3 
| | H— async@0.1.9 
| ies combined-stream®0.0.3 
1 | t— delayed-stream@0.0.5 
| “— mime@1.2.7 
t— underscore@1.4.4 
t—r request@2.14.0 
Hr form-data@0.0.7 
| H— async@@.1.22 
| “+ combined-stream@@.0.4 
| t— delayed-stream@0.0.5 
-— mime@1.2.9 





The information written out by the NPM command line contains the version of each module. In 
thepackage. | son manifest, you have a specification of which version you want to install, for 
each NPM module. For instance, we have used different version specifications such as *, 3.3.x, 
and -o.2. These follow a convention named semantic versioning, which uses three integers to 
specify the version; it consists of the major version, minor version, and patch version numbers, 
all separated by a dot. 


In the dependency list, you can specify the asterisk (*) as a wildcard that stands for any version. 
This says that your application depends on any version of that module. When you specify that 
you depend on Version 3.3.x of the nano module, any patch version of Version 3.3 will satisfy 
your requirements. The as ync version specification (-0.2) has a similar effect, saying that your 
requirements are satisfied with any patch version of 0.2. NPM will analyze what versions you 
specify and will install the latest available version that matches your specification, if available. 


After having the dependencies installed, you can use them in your code. Here is an example 
of importing some modules that we may depend on: 


var request = require('request'); 
var nano = require('nano'); 
var async = require('async'); 


Here is how Node.js resolves these dependencies when you require them. 


JJ 
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By default, NPM installs the dependencies locally inside a directory named Node modules. 
Node.js will look for this directory when you user equi re( <modul e name>) and will look 
for the module there. 





pedroteixeira® -Pro: (master) 
+ ls -la node modules 
total @ 


drwxr-xr-x 5 pedroteixeira Bass 5 
drwxr-xr-x 7 pedroteixeira 2185 o 
drwxr-xr-x 6 pedroteixeira 8 15:15 async 
drwxr-xr-x 13 pedroteixeira 8 15:15 nano 
drwxr-xr-x 14 pedroteixeira 8 15:15 request 





If the module can't be found there, Node.js will keep looking for itintheNode_modul es 
directory of the parent directory if it exists, then keep climbing up the file system tree until the 
module is found or it has reached the root directory. This means that if you have a library inside 
a nested directory in your application, it can depend on a module installed at the root of your 
application. For instance, if you create a module inside | i b/ db. j s,itcan depend onnano 
and contain code like this: 


var nano = require('nano'); 


The callback pattern 


Node.js uses the event-driven model of doing I/O, which means that every time that the current 
process has to talk to the filesystem or the network, it does so in a non-blocking manner. 


The pseudo-code for typical blocking code when doing a remote call to a database may look 
something like this: 


var result = query(' SELECT * FROM articles'); 
console. log('result:', result); 


The time that it takes between starting the query and getting back the result may take anything 
from a few milliseconds to a long time, and while that happens, your current program is just 
waiting. In order to be able to be able to do more I/O at the same time, you need to introduce 
additional execution contexts, which typically happens in the form of threads or other processes. 


In event-driven I/O, instead of returning the result of a remote operation, you pass a callback 
that gets invoked once that operation is finished. On such a platform, the equivalent of doing 
a database query similar to the previous blocking code would be: 


query('SELECT * FROM articles', function(result) { 
console. log('result:', result); 


H; 


| 
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Here, instead of relying on the return value, we're simply initiating the remote call and passing 
in a function callback as the last argument. When that remote call is completed, the function is 
invoked, passing in the result as the sole argument. 


Actually, both the blocking and non-blocking versions have been simplified, since they don't 
predict the case when an error happens. The correct blocking version would then be: 


try { 
var result = query(' SELECT * FROM articles'); 
console. log('result:', result); 
} catch(err) { 
console.error('error while performing query:', err. message); 
} 


And the equivalent event-driven version would be: 


query('SELECT * FROM articles', function(err, result) { 
if (err) { 
console.error('error while performing query:', 
err. message); 
} else { 
console. log('result:', result); 


}); 


In this last example, we're witnessing the Node.js callback pattern, which has 
two characteristics: 


+ Callback last: The callback is the last argument of the function that initiates I/O. 


+ Error first: The callback expects an error as the first argument and the results in the 
following arguments. If there is no error, the first argument isundef i ned ornull. 
If an error exists, this object should be a JavaScript error instance. 


Here is an actual working example of reading a file from disk, which you can write to a file named 
read file.js: 


var fs = require('fs'); 


fs.readFile(__filename, ‘utf8', function(err, fileContent) { 


if (err) { 

console. error(err); 
} else { 

console. log('got file content:', fileContent); 
} 


}); 


ooo 


Instant Node.js Starter 


Here, we're using the f s module, and particularly, itsr eadFi | e function to read the 
contents of the JavaScript source file from disk. We're also usingthe filename variable, 
which contains the full path for the current source file. We specify the file path and the file 
encoding (UTF-8) and pass in a function callback as the last argument. 


This last argument then gets called if there is an error or when the contents of the file are read 
from disk. 


You can run this example in the command-line prompt like this: 
$ Node read file.js 


It should output the content oftheread_file.js file. 


Chaining I/O 

When performing chaining, some I/O depends on the result of other I/O; you can chain the 
two operations by nesting the callback functions. For instance, if you want to verify the 
size of a file and only read it if it's smaller than 1 KB, you could write this into a file named 
read file conditional.js: 


var fs = require('fs'); 


fs.stat(_ filename, function(err, stats) { 





if (err) { 
console. error(err); 
} else { 
if (stats.size < 1024) { 
fs.readFile( filename, ‘utf8', function(err, fileContent) 
{ 
if (err) { 
console.error(err); 
} else { 
console.log('Got file content:', fileContent); 
} 
}); 
} else { 
console. log('Didnl't read the file, it was too long.'); 
} 


www.allitebooks.com 
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Alternatively, you can create the callback functions at the same level and name them as in this 
example(read file conditional unnested.js): 


var fs = require('fs'); 
function readFileCallback(err, fileContent) { 


if (err) { 
console. error(err); 
} else { 
console.log('Got file content:', fileContent); 
} 
} 
function statsCallback(err, stats) { 
if (err) { 
console. error(err); 
} else { 


if (stats.size < 1024) { 
fs.readFile( filename, ‘utf8', readFileCall back); 


} else { 
console.log('Didn\'t read the file, it was too 
long.'); 

} 


} 
} 


fs.stat(__filename, statsCall back); 


This will have the benefit of reducing the code indentation level but makes the execution flow a 
bit harder to follow. 


wae Alternatively, you can create or use an asynchronous flow control library such as 
Es as ync, which has some constructs that help you manage callback flow. 


Event emitter 

The callback pattern is useful when you have I/O operations that have a clear end. If you have 
an object in which events happen throughout time, the callback pattern is not a good fit. 
Thankfully, Node.js has a built-in pattern named event emitter. 
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We have already seen objects that are event emitters such as the HTTP server object, which can 
emit a series of events throughout time. Each event has a type identified by a string. Ifyou have 
an event emitter object, you can listen to any future occurrence of an event with a specific event 
type by binding a function like this: 


function eventlListener(a, b) { 
console. log('got eventl', a, b); 
} 


emitter.on('eventl', eventlListener); 


Here we're saying that every time we get an event with the event 1 type, the function named 
eventlListener should be called. Each event type can emit the event with a given set of 
arguments that will be passed on to the listener. In this case, the listener is expecting two 
arguments, a andb. 


Once you have a Listener function bound, you can remove it usingemi tter. 
removeListener, like this: 


emitter. removeListener('eventl', eventlListener); 


Besides these two listener management functions, you have also thee mi tt er. once method 
available. Using this function, you can limit the number of callbacks to a maximum of one. 


emitter.once('eventl', eventlListener); 


You can also create an event emitter by using the Node.js event emitter API. Here is an example 
of an event emitter that emits at i ck event every second: 


var EventEmitter = require('events').EventEmitter; 
var emitter = new EventEmitter(); 


var count = 0; 

seti nterval(function() { 
emitter.emit('tick', count); 
count ++; 

}, 1000); 


emitter.on('tick', function(count) { 
console. log('tick:', count); 


}); 


Here, we're using thee mi tter. emi t method to, as the name says, emit an event. The first 
argument is the event type, and the remainder of the arguments will be passed on to the 
registered event listeners (if any). 


a 
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If you save the code we just saw into a filenamedti ck emitter.js,youcanrunitlike this: 
$ Node tick_emitter.js 


The console will show a new line every second before you cancel it (using Ctrl + C): 





pedroteixeira@Pedros-MacBook-Pro: 
(master) 


+ node tick_emitter.js 

tick: 

tick: 

tick: 

tick: 

tick: 

tick: 

tick: 6 

ACpedroteixeira@Pedros-M ok-Pro: 
(master) 

all 





Parallelizing I/O 


Parallelizing I/O is natural in Node.js; you don't need to spawn new threads of execution, simply 
start two or more I/O operations in parallel. In the following example, we want to make some 
HTTP requests in parallel to http://www. twitter. com . For that, create a file named 

http requests parallel.js with the following content: 


var http require('http'); 
var urls = [ 
‘http: //search.twitter.com/search.json?q=Node', 
“http: //search.twitter.com/search. | son?q=j avascri pt' 
l; 


var allResults = []; 
var responded = 0; 


function collectResponse(res) { 
var responseBody = ''; 
res.setEncoding('utf8'); 


/// collect the response body 

res. on(' data', function(d) { 
responseBody += d; 

}); 
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/// when the response ends, we should have all the response 
body 
res.on('end', function() { 
var response = J SON. parse(responseBody); 
all Results = all Results. concat(response. results) 
console.log('l have %d results for' 
response.results. length, res.req. path) 
responded += 1; 


[ll check if we have responses to all requests 


if (responded == urls.length) { 
console.log('All responses ended. Number of total 
results:', allResults. length); 
} 

F); 


} 


urls.forEach(function(url) { 
http.get(url, collectResponse); 


DE 


Here, we are using the ht t p module to make ht t p. get requests to the Twitter search API. 
We start out by declaring which URLs you want to make requests to and then declare a callback 
function named col | ect Response. This function will be called when each of these requests 
is available. At this time, Node.js only has the response HTTP headers but doesn't yet have 

the response body. Because of that, we need to collect the body by listening to all the dat a 
events on the response that emits buffers; if you have the encoding set, it emits strings. Once 
the response ends, we can assume we have all the response body chunks and can finally 

parse it. The response is JSON-encoded, which means we can use the native) SON. parse 
function to transform the string into a JavaScript object, from which we then extract the 
results attribute. 


Since thiscol | ect Response function will be called for each individual request, we keep track 
of how many responses have ended. Once we know that we have all the responses, we say "all 
responses ended". 


At the end, we kick off this whole process by starting the requests using thehtt p. get function 
on each planned URL. 


Streams 


Node has this built-in concept of streams, which comes from the philosophy of Unix systems. 

A stream represents a source of data read or written throughout time. For instance, you can read 
the entire contents of a file at once, but if the file is large, this process will take some time and 
will consume memory. Alternatively, you can read it as a stream, where you get a piece of the 
file content at a time. 
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The readable streams 
Here is an example where you open a file read stream and listen for data chunks from it: 


var fs = require('fs'); 
var stream = fs.createReadStream('/path/to/large/file'); 


stream.on('readable', function() 1 
var chunk; 
while(chunk = stream. read()) { 
console. log('got NPM data chunk of %d bytes', 
chunk. length); 
} 
H: 


stream once('end', function() { 
console. log('stream ended'); 


}); 


Here, you can see that a read stream is an event emitter that emitsr eadabl e andend events, 
which you can listen to. When we get ar eadabl e event, we read from the stream until there 
is nothing more to read, and for each chunk of data, we print out its length. We're also logging 
stream ended to the console once we reach the end of the file. 


If you save this code into afilenamedfile stream. js,youcanrunitlike this: 
$ Node file stream js 


The console will then print out something like this: 





e090 1. bash é 


data chunk 
data chunk 
data chunk 
data chunk 
data chunk 
data chunk 
data chunk 
data chunk 
data chunk 
data chunk 
data chunk 
got data chunk 
stream ended 
pedroteixeira@Pec 


+ [] 


(master) 
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Here, we can see that we get several data chunks exactly 64 KB in size, with the exception of the 
last one. We can also see that, after the stream ended, we didn't get any more dat a events. 


The writable streams 


Besides being the source of data, a stream can alternatively or simultaneously be the target 
of data—a writable stream. You can write data out to a writable stream, and you can also end 
one. There are several examples of writable streams in Node.js. You can, for instance, create a 
writable file stream: 


var fs = require('fs'); 


var stream = fs.createWriteStream(_ dirname + '/out.txt'); 


var interval = setInterval(function() { 
stream write('tick ' + Date. now() + '\n'); 
}, 100); 


setTimeout(function() { 
clearlnterval(interval); 
stream. end(); 

+ 4950); 


Here, we're creating a file Wr i t eSt rea m and setting up two timers. The first timer is repetitive 
and fires every 100 milliseconds, and every time it fires, we compose and write out a string 
containing the current timestamp. 


The second timer only fires one after 5 seconds, and once that happens, we cancel the repeating 
interval and end the write stream. 


If you save this code example into afilenamedwritable file stream. js, youcanrunit 
like this: 


$ Node writable file_streamjs 
After 5 seconds, your Node process should end, and you should have a fle namedout. txt 


containing 931 bytes of text. 


The duplex streams 


A stream can be both writable and readable at the same time, permitting both obtaining data 
from it and writing data to it. An example of such a stream can be a TCP connection: 


var net = require('net'); 
var server = net.createServer(); 


server. on('connection', function(stream) { 
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stream. setEncoding('utf8'); 
console. log('got new stream ); 


stream. on('readable', function() { 


var buf; 
while(buf = stream read()) { 
console.log(' data: ', buf); 


stream. write(buf.toUpperCase()); 
} 
}); 


stream. on('end', function() { 
console. log('stream ended' ); 


}); 
ae 


server. listen( 8080); 


Here, we're creating a TCP server, which we will bind to port 8080. On every connection we need 
to do the following: 


+ Set the string encoding so that we get UTF-8-encoded strings instead of buffer chunks 
when we read data from it. 


+ Addalistener forthe readabl e events where we fetch all the available data from the 
stream. Every time we get a new data chunk, we do the following: 


o 


Print out the received data 


o 


Transform the string into uppercase 


o 


Write that string back into the stream 


You can start this server by saving the previous code into a file namedt cp server. js and 
then starting it like this: 


$ Node tcp server.js 


In a different console window, connect to the server using a command-line utility, such as 
Tel net ornc. Here, I'm usingnc to connect to localhost port 8080: 


$ nc localhost 8080 
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Now you can type text into the remote session, and the server should respond with that text 
in uppercase: 





80900 2. nc a 
Last login: Mon Mar 11 14:56:38 on ttys001 
pedroteixeira@Pedros-Ma k-Pro: 

+ nc localhost 8080 

pass me the salt please 

PASS ME THE SALT PLEASE 


hello!! 


HELLO! ! 





Stream flow control 


In Node.js, streams are not only about sending and receiving data but also about controlling the 
flow of that data. When you write data into a writable stream, that operation is non-blocking, 
which means that the data may get stored somewhere before it's actually sent out to the 
underlying resource. For instance, if you're writing to a file or a network connection, Node.js 
may not be able to send that data into the kernel while the kernel buffers still hold previously 
sent data. 


In Node.js streams, the calltostream. write willreturnabool ean value saying whether the 
buffer was flushed to the underlying resource. If the wr i te operation returns f al se, the stream 
will later emit adr ai n event once the buffer gets flushed. We can test that by putting the 
following codeinafilenamedwrite drain. js: 


var fs = require('fs'); 
var stream = fs.createWriteStream(_ dirname + '/out.txt'); 


for(var | = 0; i < 50000; i++) { 
if (! stream write(i.toString() +' ')) { 
console.log('not flushed'); 
} else { 
console. log('flushed'); 
} 
} 


console. log('wrote all'); 
stream. on('drain', function() { 


console. log('drained'); 


}); 


CO 
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Then, run it: 


$ Node write drain.js 





890 1. bash ar 


flushed 
flushed 
flushed 
flushed 
flushed 
flushed 
flushed 
flushed 
flushed 
flushed 
flushed 
flushed 
wrote all 
drained 





You will observe several not flushed messages, followed by a wrote all message, followed by 
a drained message. This means that none of the wr i t e operations were immediately flushed; 
instead, they got queued. Once the Node manages to flush all that data, itemits adr ai n event. 


Node.js also has API support for flow control on the readable streams. We have already 
observed that, when data is available onar eadabl e stream, ther eadabl e event is emitted. 
This makes the data buffer up on ther eadabl e stream before we read it. Asinthewri table 
streams, as data gets buffered, memory gets consumed and will eventually run out. Fortunately, 
ther eadabl e streams have a built-in flow control mechanism as it ceases from pulling data 
from the underlying resource if a high wat er mar k is reached. This high watermark, which can 
be configured inthest ream. Readabl e constructor, defines the maximum number of bytes 
to store before pausing. Also, some concrete implementations of ther eada bl e streams allow 
you to change this value, as in the example of fs. creat eReadStream: 


var fs = require('fs'); 
var stream = fs.createReadStream('/path/to/file/', {highWaterMark: 
400003); 


Piping 
If youhaveareadabl e andwritabl e stream you can connect one to the other using the 
pipe mechanism: 


var sourceStream 
var targetStream = .. 
sourceStream. pipe(target Stream); 
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Here, every time there is data available on the source stream, it gets read and written to the 
target stream. Also, it is possible to unpipe a destination stream from the source stream using 
stream. unpipe: 


sourceStream.unpipe(targetStream); 


Next is an example of an echo server that pipes all the data coming from a connection back into 
that connection: 


var net = require('net'); 
var server = net. createServer(); 


server. on('connection', function(stream) { 
stream. pi pe(stream); 
stream. pipe( process. stdout); 


H; 
server.listen(8080); 


Here, we're setting up a TCP server listening on port 8080. Every time the server gets a new 
connection, we pipe the connection stream into itself and also into the process standard output. 
The first pipe has the effect of writing back every bit of data that any client sends, making this 
server effectively an echo server. The second pipe is writing every bit of data that one client 
sends into the process output, which is a writable stream. 


If you save the previous code block in a fle namedecho_server. j s,runitand connect it toa 
command-line utility such as Tel net ornc, you will notice that the client receives every bit of 
data that is sent to the server. 


The transform streams 

Since the calltost ream. pi pe returns the target stream, it is possible to chain pipes: 
var sourceStream =... 
var transformStream = .. 


var targetStream = .. 
sourceStream.pipe(transformStream).pipe(targetStream); 


This last line pipes every bit of data cominginfroms our ceStreamintotransformStream 
and every bit of data coming out oftransformStreamintotarget Stream. 


This can be useful when you need to react to incoming data and transform a stream into another 
stream. For instance, we can implement our previous uppercasing TCP server example into a 
Transf or m stream like this: 


var Transform = require('stream ). Transform; 


var server = require('net').createServer(); 
server.on('connection', function(stream) { 
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stream. setEncoding('utf8'); 


var uppercase = new Transform( (decodeStrings: false}); 
uppercase. transform = function(d, encoding, cb) { 
cb(null, d.toUpperCase()); 

F 


stream. pipe( uppercase). pipe(stream); 


iE 
server.listen(8080); 


Here, we're using thestr eam. Transf or m utility class from the Node core API to create a 
Transf orm stream. This stream has to implementa_transf or m method that performs the 
actual transformation. This function receives the callback as the third argument, which it has to 
call using the callback pattern: error first and result last. In our case, we can perform a callback 
immediately since the transformation is a synchronous operation, passing in nul | asthe error 
argument to signal that no error occurred during the transformation. 


You can save this last piece of code into aflenamedupper case server stream. js and 
run it: 


$ Node uppercase_server_stream. js 


You can then, from another window, use Tel net, nc, or a similar command-line tool to open a 
connection to the server. In this example, I'm using nc : 





eoo 1. nc i 
node ne 
+ nc localhost 8080 


today is a beautiful day 
TODAY IS A BEAUTIFUL DAY 











These are the main concepts that will get you started in the Node.js world. The callback pattern, 
the event emitter, and the streams are the building blocks of the Node.js core API and also of an 
ecosystem of open-source modules contained in NPM. Since the Node.js core API is a low-level 
one, you will almost surely require using third-party modules to build an interesting application. 
With the knowledge of how these patterns work and with the help of some of the Node.js 
community resources, you will be able to understand and use them to accomplish your goals. 
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People and places you should get to know 


Official sites 


The official Node.js website, containing links to the downloads, API documentation, blog, 
and other interesting resources is as follows: 


http://Nodejs.org 


Blogs 


This is where the Node.js core development team posts updates and plans. 


http://blog.Nodejs.org 


API documentation 
The central resource for accessing documentation on the Node.js core API is as follows: 


http://Nodejs.org/api/ 


NPM modules 


You can use it to search for packages and to show package documentation and metadata. 
The web frontend of the NPM repository is as follows: 


https://npmjs.org 


Github repository 


The versioned code repository where the Node.js core development happens is as follows: 


https://github.com/j oyent/ Node 


Community 
+ Mailing list: http: // groups. google. com/group/ Nodej s 
This is the mailing list for the Node.js core, where you can post questions that can be 
answered by the community. 


+ IRC channels are as follows: 
o 


Server:irc.freeNode. net 


° Channel: #Nodej s 
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This is a highly frequented IRC channel where you can post questions and answers 
about all things Node.js. 

+ Stack Overflow Node.jstag: http: /Istackoverflow. com questions/tagged/ Node. js 
All questions posted on Stack Overflow are tagged with Node.js. 


+ Nodeup podcast: http: // Nodeup. com 
This has frequent podcasts hosted by the Node.js core team members and some 
Node.js community leaders. 

+ Nodetuts screencast tutorials: http://Nodetuts.com 


This has free screencast tutorials about Node.js. 
+ Node.js conferences: 


° Node Conf (USA): ht tp: // www. Nodeconf. com 
° Node Dublin (Ireland): ht tp://Nodedublin. com 


Twitter 

+ Isaac Schlueter (the Node.js development leader): 
https://twitter.comizs 

+ James Halliday (author of the many Node.js modules, guides, and products; co-creator 
of browserling and testling): 
https://twitter.com/substack 

+ Mikeal Rogers (organizer of the NodeConf; Node.js core team member; author of the 
request NPM module): 
https://twitter.com/ mi keal 

+ Charlie Robbins (author of many Node.js NPM modules; CEO of Nodejitsu, a PaaS 
provider of Node.js): 


ps://twitter.com/indexzero 


+ Paolo Fragomeni (author of many popular Node.js NPM modules): 


ps://twitter.com/hij lnx 


+ Bert Belder (Node.js core and LibUV developer; co-founder of Strongloop, a Node.js): 

















ps://twitter.com/piscisaureus 


+ Tim Caswell (freelancer; author of many Node-related open source projects): 











https://twitter.com/creationix 
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Daniel Shaw (co-founder of The Node Firm, a consulting company specializing in Node. 
js; an engineer working at Voxer): 


https://twitter.com/dshaw 


Elijah Insua (hardware hacker; author of the famous JSDOM NPM module): 
https://twitter.com/tmpvar 


Nuno Job (geek, open-source enthusiast, and shapes the future of Node.js at 
Nodejitsu): 


https://twitter.com/dscape 
Pedro Teixeira: 


https://twitter.com/pgte 
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